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Abstract 


Visualizing  uncertainty  can  be  a  challenging  endeavour.  In  an  attempt  to  minimize  the 
challenges,  this  paper  defines  a  systematic  approach  to  designing  a  visual  representation  of 
uncertainty  called  the  Uncertainty  Visualization  Development  Strategy  (UVDS).  The  strategy 
helps  in  the  understanding  of  both  the  data  and  the  uncertainty.  The  UVDS  has  eleven  steps 
which  include:  identify  the  uncertainty  visualization  task;  understanding  the  data  that  need  to 
have  their  uncertainty  visualized;  understanding  why  uncertainty  needs  to  be  visualized  and  how 
the  uncertainty  visualization  needs  to  help  the  user;  deciding  on  the  uncertainty  to  be  visualized; 
deciding  on  a  definition  of  uncertainty;  determining  the  specific  causes  of  the  uncertainty; 
determining  the  causal  categories  of  the  uncertainty;  determining  the  visualization  requirements; 
calculating,  assigning,  or  extracting  the  uncertainty;  trying  different  uncertainty  visualization 
techniques;  and  obtaining  audience  opinions  and  criticisms.  The  UVDS  has  been  created 
specifically  to  help  the  designer  produce  comprehensive  uncertainty  visualizations,  allow  the 
designer  more  time  to  focus  on  the  creative  aspects  of  the  work,  and  give  those  trying  to 
understand  what  is  behind  the  design  a  clearer  understanding.  As  an  example  application  of  the 
UVDS,  it  is  applied  to  current  research  regarding  uncertainty  visualization  for  the  Canadian 
Recognized  Maritime  Picture  (RMP). 


Resume 


Visualiser  T incertitude  peut  etre  une  entreprise  extremement  difficile.  Afm  de  minimiser  les 
difficultes,  le  document  ci-joint  propose  une  approche  systematique  pour  la  conception  d’une 
representation  visuelle  de  T incertitude  :  la  Strategic  de  visualisation  de  T incertitude  (SVI).  Cette 
strategic  aide  a  comprendre  les  donnees  et  Tincertitude  qui  s’y  rattache.  La  SVI  comporte 
1 1  etapes  :  definir  la  tache  de  visualisation  de  Tincertitude;  comprendre  les  donnees  dont 
Tincertitude  doit  etre  visualisee;  comprendre  pourquoi  Tincertitude  doit  etre  visualisee  et 
comment  cette  visualisation  aidera  Tutilisateur;  decider  de  Tincertitude  a  visualiser;  donner  une 
definition  de  Tincertitude;  determiner  les  causes  precises  de  Tincertitude;  determiner  les 
categories  causales  de  Tincertitude;  determiner  les  besoins  en  matiere  de  visualisation;  calculer, 
attribuer  ou  extraire  Tincertitude;  essayer  differentes  techniques  de  visualisation  de  Tincertitude; 
et  recueillir  des  opinions  et  des  commentaires  critiques.  La  SVI  a  ete  creee  specifiquement  pour 
aider  le  concepteur  a  produire  une  visualisation  complete  de  Tincertitude,  pour  permettre  au 
concepteur  de  consacrer  plus  de  temps  aux  aspects  creatifs  du  travail,  et  pour  aider  les  interesses  a 
mieux  comprendre  le  systeme  de  visualisation.  Exemple  d’ application  :  la  SVI  est  utilisee  dans 
les  recherches  en  cours  sur  la  visualisation  de  Tincertitude  pour  le  Tableau  de  la  situation 
maritime  (TSM)  du  Canada. 
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Executive  summary 


A  Strategy  for  Uncertainty  Visualization  Design 

Anna-Liesa  S.  Lapinski;  DRDC  Atlantic  TM  2009-151;  Defence  R&D  Canada  - 
Atlantic;  October  2009. 

Introduction:  Achieving  effects-based  visualization  and  awareness  for  the  decision  maker  is  an 
important  area  of  modem  defence  research.  Making  effects-based  decisions  in  the  defence 
environment  often  involves  relying  on  information  from  many  sources  such  as  sensors  and  human 
intelligence.  In  order  to  make  an  educated  decision  (effects-based  or  otherwise),  it  is  important  to 
understand  the  uncertainty  in  the  information  being  used  to  make  that  decision;  however,  the 
visual  representation  of  uncertainty  is  often  overlooked  in  visualizations.  A  decision  maker 
cannot  form  a  clear  awareness  of  the  situation  without  understanding  the  inherent  uncertainty  in 
the  information  that  is  being  visualized. 

While  it  is  a  well  defined  research  area  within  the  topic  of  information  visualization,  uncertainty 
visualization  is  lacking  proven  (human  factors)  theory,  particularly  in  the  defence  domain,  to  help 
those  who  need  to  develop  uncertainty  visualizations.  In  an  attempt  to  minimize  the  challenges 
encountered  in  uncertainty  visualization,  this  paper  defines  a  systematic  approach  to  designing  a 
visual  representation  of  uncertainty  called  the  Uncertainty  Visualization  Development  Strategy 
(UVDS).  The  strategy  helps  in  the  understanding  of  both  the  data  and  the  uncertainty. 

Results:  The  UVDS  that  is  presented  has  eleven  steps  which  include:  identify  the  uncertainty 
visualization  task;  understanding  the  data  that  need  to  have  their  uncertainty  visualized; 
understanding  why  uncertainty  needs  to  be  visualized  and  how  the  uncertainty  visualization  needs 
to  help  the  user;  deciding  on  the  uncertainty  to  be  visualized;  deciding  on  a  definition  of 
uncertainty;  determining  the  specific  causes  of  the  uncertainty;  determining  the  causal  categories 
of  the  uncertainty;  determining  the  visualization  requirements;  calculating,  assigning,  or 
extracting  the  uncertainty;  trying  different  uncertainty  visualization  techniques;  and  obtaining 
audience  opinions  and  criticisms.  The  UVDS  can  help  the  visualization  designer  produce 
comprehensive  uncertainty  visualizations,  allow  the  designers  more  time  to  focus  on  the  creative 
aspects  of  the  work,  and  give  those  trying  to  understand  what  is  behind  the  design  a  clearer 
understanding.  As  an  example  application  of  the  UVDS,  it  is  applied  to  current  research 
regarding  uncertainty  visualization  for  the  Canadian  Recognized  Maritime  Picture  (RMP). 

Significance:  Decision  making  quality  is  improved  from  understanding  the  uncertainty  in  the 
data  and  information  being  used  in  the  decision  process.  For  example,  an  RMP  with  none  of  the 
uncertainty  in  its  contacts  displayed  looks  like  all  the  information  is  current,  all  the  ships  are 
currently  in  the  positions  displayed  on  the  map,  all  the  information  comes  from  sources  that  never 
have  mistakes,  and  the  entire  area  is  monitored  by  sensors  feeding  the  RMP.  For  an  informed 
decision  to  be  made  regarding  the  RMP,  whether  it  is  regarding  where  to  send  a  surveillance 
asset,  how  to  prioritize  the  order  a  group  of  ships  should  be  investigated,  etc.,  more  information 
on  the  uncertainty  surrounding  the  contacts  in  the  RMP  needs  to  be  known.  While  the  uncertainty 
levels  can  be  determined  by  investigating  each  contact  one  by  one,  that  is  a  time  consuming 
process  if  there  are  more  than  a  few  ships  to  be  investigated.  Having  a  visual  representation  of 
the  uncertainty  overlaid  onto  the  RMP  can  help  a  decision  maker  quickly  assess  a  situation  and,  if 


iii 


DRDC  Atlantic  TM  2009-151 


necessary,  decide  on  which  contacts  or  areas  are  worth  a  closer  look  before  a  decision  is  made. 
This  simple  development  strategy  for  creating  uncertainty  visualizations  can  be  applied  to  any 
situation  (not  just  defence  related)  where  uncertainty  needs  to  be  visualized. 


Future  plans:  This  is  the  first  of  a  series  of  Technical  Memorandums  (TMs)  aimed  at  creating 
foundational  documents  on  the  topic  of  uncertainty  visualization  which  can  be  used  in  defence 
applications.  In  addition,  the  aim  of  these  TMs  will  be  to  push  forward  the  research  of  uncertainty 
visualization,  particularly  in  the  defence  field. 
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A  Strategy  for  Uncertainty  Visualization  Design 

Anna-Liesa  S.  Lapinski;  DRDC  Atlantic  TM  2009-151;  R  &  D  pour  la  defense 

Canada  -  Atlantique;  Octobre  2009. 

Introduction  :  Foumir  aux  decideurs  des  outils  de  visualisation  et  de  connaissance  de  la  situation 
est  un  secteur  important  de  la  recherche  pour  la  defense.  Pour  prendre  des  decisions  basees  sur  les 
effets,  les  specialistes  de  la  defense  doivent  souvent  se  fier  a  des  informations  provenant  de 
nombreuses  sources,  comme  les  capteurs  et  1’ intelligence  humaine.  Pour  prendre  une  decision 
eclairee  (basee  sur  les  effets  ou  autre),  il  est  important  de  connaitre  le  degre  d’incertitude  des 
informations  utilisees  dans  la  prise  de  decision.  Cependant,  la  representation  visuelle  de 
l’incertitude  est  souvent  negligee  par  les  systemes  de  visualisation.  Un  decideur  ne  peut  pas  avoir 
une  vision  claire  de  la  situation  sans  connaitre  le  degre  d’incertitude  des  informations  visualisees. 

Bien  qu’il  s’agisse  d’un  champ  de  recherche  bien  defini  lie  a  la  visualisation  de  1’ information,  la 
visualisation  de  l’incertitude  souffre  de  l’absence  d’une  theorie  eprouvee  (facteurs  humains), 
surtout  dans  le  domaine  de  la  defense.  Afm  de  minimiser  les  difficultes  liees  a  la  visualisation  de 
l’incertitude,  le  document  ci-joint  propose  une  approche  systematique  pour  la  conception  d’une 
representation  visuelle  de  1’ incertitude  :  la  Strategic  de  visualisation  de  l’incertitude  (SVI).  Cette 
strategic  aide  a  comprendre  les  donnees  et  l’incertitude  qui  s’y  rattache. 

Resultats  :  La  SVI  comporte  1 1  etapes  :  definir  la  tache  de  visualisation  de  l’incertitude; 
comprendre  les  donnees  dont  l’incertitude  doit  etre  visualisee;  comprendre  pourquoi  l’incertitude 
doit  etre  visualisee  et  comment  cette  visualisation  aidera  l’utilisateur;  decider  de  l’incertitude  a 
visualiser;  donner  une  definition  de  l’incertitude;  determiner  les  causes  precises  de  l’incertitude; 
determiner  les  categories  causales  de  l’incertitude;  determiner  les  besoins  en  matiere  de 
visualisation;  calculer,  attribuer  ou  extraire  l’incertitude;  essayer  differentes  techniques  de 
visualisation  de  l’incertitude;  et  recueillir  des  opinions  et  des  commentaires  critiques.  La  SVI  peut 
aider  le  concepteur  a  produire  une  visualisation  complete  de  1’ incertitude,  permettre  au 
concepteur  de  consacrer  plus  de  temps  aux  aspects  creatifs  du  travail,  et  aider  les  interesses  a 
mieux  comprendre  le  systeme  de  visualisation.  Exemple  d’application  :  la  SVI  est  utilisee  dans 
les  recherches  en  cours  sur  la  visualisation  de  l’incertitude  pour  le  Tableau  de  la  situation 
maritime  (TSM)  du  Canada. 

Portee  :  Pour  ameliorer  la  prise  de  decision,  il  faut  connaitre  le  degre  d’incertitude  des  donnees  et 
des  informations  utilisees  dans  le  processus  de  prise  de  decision.  Par  exemple,  dans  un  TSM,  s’il 
n’y  avait  aucune  indication  sur  Tincertitude  associee  aux  contacts,  on  pourrait  croire  que  toutes 
les  donnees  affichees  sont  a  jour,  que  tous  les  navires  sont  actuellement  a  l’emplacement  indique, 
que  toutes  les  informations  proviennent  de  sources  qui  ne  font  jamais  d’erreurs,  et  que  l’ensemble 
du  secteur  est  couvert  par  des  capteurs  qui  «  nourrissent »  le  TSM.  Pour  prendre  des  decisions 
eclairees  a  l’egard  du  TSM  (ex.  :  determiner  ou  sera  installe  un  equipement  de  surveillance, 
etablir  l’ordre  dans  lequel  les  navires  seront  examines,  etc.),  il  faut  avoir  plus  d’informations  sur 
Tincertitude  associee  aux  contacts.  Le  niveau  d’incertitude  peut  etre  determine  en  examinant  tous 
les  contacts  un  a  un,  mais  c’est  un  processus  qui  prend  trap  de  temps  s’il  y  a  plus  que  quelques 
navires  a  examiner.  Lorsqu’une  representation  visuelle  de  Tincertitude  est  ajoutee  au  TSM,  le 
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decideur  peut  evaluer  rapidement  la  situation,  et  decider  si  necessaire  des  contacts  ou  des  secteurs 
qui  meritent  d’etre  examines  plus  attentivement  avant  qu’une  decision  soit  prise.  Cette  strategic 
toute  simple  de  visualisation  de  1’  incertitude  peut  etre  appliquee  a  toutes  les  situations  (pas 
seulement  dans  le  domaine  de  la  defense)  ou  l’incertitude  doit  etre  visualisee. 

Recherches  futures  :  Ceci  est  le  premier  d’une  serie  de  memoires  techniques  (MT)  confus 
comme  des  documents  de  base  sur  la  visualisation  de  l’incertitude,  et  destines  aux  applications  de 
la  defense.  Ces  MT  ont  aussi  pour  objectif  de  faire  progresser  la  recherche  sur  la  visualisation  de 
Tincertitude,  surtout  dans  le  domaine  de  la  defense. 
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1  Introduction 


Visualizing  uncertainty  is  a  challenging  endeavour.  Perhaps  this  is  because  uncertainty  is  always 
calculated  or  recorded  with  respect  to  other  primary  data.  In  a  sense,  uncertainty  is  more  like 
metadata  to  primary  data.  Thus,  the  act  of  visualizing  the  uncertainty  has  to  be  supported  by  an 
understanding  of  both  the  uncertainty  itself  and  the  primary  data.  The  primary  data  and  how 
these  data  are  currently  being  used  act  as  two  constraints  on  the  visualization  solution.  It  is 
challenging  to  develop  innovative  methods  to  visualize  uncertainty  given  these  innate  constraints; 
however,  many  have  begun  to  foray  into  this  research  area  [1-12]. 

As  of  yet,  proven  theory  for  visualizing  uncertainty  is  lacking  in  the  literature.  Some  guidance 
can  be  found  in  the  general  visualization  theory  of  Tufte  [13],  Bertin  [14],  Ware  [15],  and 
Chambers  et  al.  [16].  Their  theories  have  been  applied  in  the  literature  to  assess  uncertainty 
visualizations  [17,  18]  as  well  as  to  guide  designs  [5].  As  much  of  uncertainty  visualization 
requires  the  presentation  of  multidimensional  data,  Cluffs  [19]  critiques  which  draw  on  Bertin 
[14]  and  Chambers  et  al.  [16]  are  also  worth  noting.  Some  guidance  can  also  be  found  by 
surveying  the  literature  for  trends.  One  trend,  for  example,  is  that  the  most  popular  technique  for 
visualizing  uncertainty  is  to  integrate  the  uncertainty  data  into  a  visualization  of  the  primary  data 
or  spatial  coordinates  of  the  primary  data  (e.g.,  [4-8]).  With  the  absence  of  solid,  experimentally 
proven,  uncertainty  visualization  theory,  the  uncertainty  visualization  designer  is  left  to  use 
educated  guesses  augmented  by  their  own  creativity,  to  create  their  uncertainty  visualization. 

This  paper  attempts  to  make  visualizing  uncertainty  somewhat  less  challenging  by  outlining  a 
systematic  approach,  called  the  Uncertainty  Visualization  Development  Strategy  (UVDS),  for 
designing  an  uncertainty  visualization.  The  UVDS  is  an  eleven  step  strategy  for  visualizing 
uncertainty.  Each  step  is  explained  and  described  in  this  paper.  The  goal  of  the  UVDS  is  to  guide 
the  visualization  designer  such  that  the  uncertainty  visualization  that  is  developed  suits  the 
primary  data,  the  uncertainty  being  visualized  and  the  visualization  user’s  needs. 

The  objective  of  this  paper  is  to  document  a  step  by  step  practical  method  for  creating  uncertainty 
visualizations.  The  user  of  the  UVDS  will  still  be  required  to  make  educated  guesses,  be  creative 
and  be  aware  of  the  primary  data,  but  by  following  this  systematic  approach,  many  of  the  easily 
forgotten  but  important  details  will  be  systematically  accomplished.  The  strategy  should  leave  the 
user  more  time  to  focus  on  the  creative  aspect  of  the  visualization.  In  addition,  when 
documenting  work  on  a  new  visualization,  reporting  on  each  step  in  the  development  strategy 
should  help  the  audience  better  understand  the  work.  For  those  who  are  already  trying  to 
visualize  a  particular  uncertainty,  implementing  the  UVDS  will  help  clarify  the  underlying  goals 
of  the  effort  and  identify  omitted  steps  that  might  be  hampering  progress. 

This  strategy  can  be  used  in  conjunction  with  previously  developed  visualization  theory, 
postulates  or  frameworks  that  aim  to  guide  creative  or  functional  aspects  when  creating  an 
optimal  visualization.  The  UVDS  does  not  tell  the  user  how  to  create  a  visualization,  how  to 
optimize  the  human-visualization  interaction,  or  what  the  user  and  visualization  requirements  are, 
but  rather  it  provides  the  user  with  an  opportunity  to  address  such  things.  The  UVDS  is  a 
foundation  onto  which  existing  visualization  work  can  be  applied.  Such  work  could  include  the 
previously  mentioned  Tufte  [13],  Bertin  [14],  Ware  [15],  and  Chambers  et  al.  [16],  as  well  as 
Amar  and  Stasko  [20],  Zuk  and  Carpendale  [21],  and  Kreuseler  et  al.  [22],  to  name  a  few.  There 
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will  likely  be  occasion,  such  as  in  the  framework  developed  by  Amar  and  Stasko  [20]  to  bridge 
analytic  gaps  in  visualization  design,  when  tasks  in  the  strategy  overlap  (and  therefore 
complement)  design  frameworks. 

This  paper  presents  a  simple  development  strategy  for  developing  uncertainty  visualizations.  In 
the  following  section,  the  steps  of  the  UVDS  are  discussed  and  defended.  In  Section  3,  ongoing 
work  regarding  uncertainty  visualizations  for  the  (Canadian)  Recognized  Maritime  Picture  (RMP) 
is  used  to  illustrate  the  completeness  of  the  UVDS.  Final  remarks  are  made  in  the  remaining 
section. 
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2  Uncertainty  Visualization  Development  Strategy 
(UVDS) 


The  UVDS  represents  a  series  of  steps,  recipe,  or  process  to  be  applied  when  an  uncertainty 
visualization  needs  to  be  designed.  Such  recipes  are  important  for  multi-step  processes,  because 
they  help  guide  the  designer  and  ensure  steps  of  the  process  are  not  forgotten.  The  UVDS  is 
illustrated  in  Figure  1.  The  first  step  requires  the  designer  to  identify  the  uncertainty  visualization 
task.  This  needs  to  be  accomplished  in  only  vague  terms  as  much  of  the  strategy  will  help 
illuminate  the  intricacies  of  the  problem.  For  example,  stating  “the  uncertainty  of  [such-and- 
such]  data  needs  to  be  visualized  for  [so-and-so,  or  such-and-such  website,  or  such-and-such 
application,  etc.],”  could  be  sufficient  to  accomplish  this  step.  This  is  an  obvious  first  step  but  is 
included  to  act  as  a  guiding  statement  for  the  remaining  work. 

The  second  step  is  also  quite  straightforward  and  would  hopefully  always  be  done  automatically. 
However,  the  degree  to  which  the  step  is  completed  is  also  critical.  To  understand  the  data  that 
need  to  have  their  uncertainty  visualized  ensures  that  the  person  designing  the  visualization 
understands  the  primary  data  to  which  the  uncertainty  is  related.  It  is  useful,  for  example,  to  know 
the  origin  of  the  data;  to  understand  the  units  associated  with  the  data;  to  understand  the  precision 
of  numerical  values;  to  understand  the  precision  to  which  something  is  being  visually  rendered  or 
reconstructed;  to  know  if  the  data  being  visualized  have  been  modified  (e.g.,  averaged);  to  know 
if  the  data  are  continuously  being  generated  or  if  the  data  set  is  static;  to  know  the  data  format  and 
limitations  introduced  by  the  format;  etc.  Perhaps  more  difficult  but  equally  important,  the  user 
should  understand  any  intricate  relationships  that  exist  within  the  data  (e.g.,  how  changes  in  one 
type  of  data  influence  other  data).  This  step  in  the  strategy  can  make  the  subsequent  steps  easier 
and  may  illuminate  some  constraints  on  the  uncertainty  visualization. 

To  understand  why  uncertainty  needs  to  be  visualized  is  essential  to  creating  an  effective 
uncertainty  visualization.  In  some  cases,  this  understanding  will  highlight  the  fact  that  the 
same  data  should  be  visualized  differently  for  different  audiences.  For  example,  a  designer 
representing  positional  uncertainty  of  in-flight  airplanes  for  an  air  traffic  control  officer  will  take 
a  different  approach  to  uncertainty  visualization  as  compared  to  presenting  the  same  data  to  a 
group  of  scientists  who  are  studying  plane  traffic  patterns  and  the  related  uncertainties.  In  these 
cases,  the  requirements  for  the  visualizations  are  different;  one  case  is  dominated  by  safety  while 
the  other  dominated  by  a  statistical  representation  of  the  observations.  In  this  step  the  designer 
must  therefore  comprehend  how  the  uncertainty  visualization  needs  to  help  the  user  (even  if  they, 
themselves,  are  the  user). 

The  next  part  of  the  strategy  has  four  steps  which  have  been  grouped  together  in  Figure  1  to 
indicate  that  the  steps  are  interconnected  and  may  be  completed  in  any  order.  For  one  step  in  the 
group,  the  designer  has  to  decide  on  the  uncertainty  to  be  visualized.  This  is  important  because 
there  may  be  multiple  uncertainties  associated  with  the  data.  For  this  step  the  designer  needs  to 
recognize  the  uncertainties  that  could  be  visualized  and  then  narrow  these  down  to  the 
uncertainties  that  should  be  visualized.  Selecting  the  proper  uncertainty  to  be  visualized  is  critical 
to  meeting  user  requirements.  This  step  is  not  placed  earlier  in  the  strategy  because  the  earlier 
steps  may  result  in  showing  that  the  preconceptions  the  designer  had  at  the  beginning  of  the 
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Figure  1:  A  graphical  (“waterfall”)  representation  of  the  UVDS. 
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UVDS  concerning  what  uncertainty  needed  to  be  visualized  were  not  fully  understood  at  that 
time. 

An  essential  step  within  this  group  is  for  the  designer  to  decide  on  a  definition  of  uncertainty. 
The  definition,  once  formalized,  may  indicate  that  the  original  preconception  of  the  uncertainty  to 
be  visualized  is  too  broad  (or  restricting)  and,  therefore,  either  the  definition,  the  uncertainty  to  be 
visualized,  or  both  will  need  to  be  revised.  In  addition,  in  the  uncertainty  visualization  literature 
it  is  common  for  authors  to  comment  or  show  that  there  are  many  definitions  of  uncertainty  [7, 
10-12,  18].  The  existence  of  multiple  definitions  can  create  misunderstanding  among  designers 
and  users.  The  designer  cannot  and  should  not  assume  everyone  is  working  from  the  same 
definition  of  uncertainty.  Again,  this  step  is  not  placed  earlier  in  the  strategy  because  the 
increased  understanding  of  the  problem  that  the  second  and  third  step  of  this  strategy  brings  to  the 
designer  will  feed  into  defining  what  is  meant  by  uncertainty.  The  designer  may  feel  more 
comfortable  doing  this  step  before  they  decide  on  the  uncertainty  to  be  visualized  which  is,  as 
mentioned  earlier,  why  this  area  of  Figure  1  is  non-linear. 

The  designer  should  also  determine  the  specific  causes  of  the  uncertainty.  For  example,  consider 
the  position  of  a  ship  received  every  30  minutes.  Between  position  fixes,  one  cause  of 
uncertainty  in  knowing  the  ship  position  is  the  time  lag  between  position  reports.  Note  that  this 
has  nothing  to  do  with  positional  accuracy  or  precision,  but  rather  is  related  to  the  time-late  nature 
of  the  data  delivery  to  the  user.  Isolating  the  actual  cause  of  the  uncertainty  can  illuminate  the 
need  to  group  or  ungroup  particular  uncertainties  when  visualizing.  This  step  (and  also  the  next 
step)  helps  to  ensure  that  an  uncertainty  that  needs  to  be  included  in  the  visualization  is  not  being 
overlooked.  Determining  the  specific  causes  of  uncertainty  can  often  illuminate  important 
uncertainties  not  previously  recognized  or  additional  components  of  the  uncertainty  of  interest. 
These  may  need  to  be  incoiporated  into  the  calculations  or  visualization.  For  example,  in  the 
previous  ship  example,  there  is  also  uncertainty  associated  with  the  positional  data  itself  (i.e.,  the 
precision  and  accuracy  of  the  positional  data).  Realizing  the  multiple-component  nature  of  the 
uncertainty  may  require  a  revisit  to  the  two  steps  described  previously. 

It  is  also  useful  to  determine  the  causal  categories  of  the  uncertainty,  or  in  other  words  the 
categories  the  specific  causes  fall  into.  Table  1  lists  the  causal  categories,  developed  from  related 
lists  by  Plewe  [23]  and  Griethe  and  Schumann  [10].  The  most  important  reason  to  determine  the 
causal  categories  is  to  illuminate  what  types  of  uncertainty  the  designer  is  dealing  with.  This 
information  will  be  valuable  when  deciding  how  to  visualize  the  uncertainty;  e.g.,  uncertainty  due 
to  data  age  may  need  to  be  represented  differently  than  uncertainty  due  to  known  data  corruption. 
In  addition,  sometimes  it  can  be  easier  to  think  of  the  broad  causal  categories  before  narrowing 
down  the  specific  causes  in  the  previous  step.  Again,  completing  this  step  may  require  the 
designer  to  revisit  the  previous  steps  in  this  grouping  illustrated  in  Figure  1,  perhaps  because  they 
realize  their  definition  is  too  narrow  or  the  identified  uncertainty  to  be  visualized  is  incomplete. 

At  this  point  in  the  design  strategy,  the  focus  shifts  to  the  visualization  itself.  To  determine  the 
visualization  requirements  the  designer,  from  a  visualization  point  of  view,  identifies  the  needs  of 
the  visualization.  For  example,  if  the  uncertainty  is  being  visualized  concurrently  with  the 
primary  data,  is  there  a  requirement  for  the  uncertainty  visualization  to  be  the  dominant  feature? 
Another  question  of  requirement  would  be  at  what  level  of  detail  the  audience  should 
comprehend  the  uncertainty:  e.g.,  the  audience  simply  knows  that  there  is  uncertainty;  the 
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Table  1:  Causal  Categories  of  Uncertainty. 


Causal  Categories 
of  Uncertainty 

Definition 

Data  Acquisition 
Limitation 

The  limitation  caused  by  the  accuracy  and  precision  of  the  data  acquisition 
device,  including  the  human  senses  and  mind. 

Data  Acquisition 
Error 

An  error  in  a  datum  acquisition  that  occurs  at  the  time  of  acquisition.  The 
error  can  be  accidental  or  deliberate.  The  resulting  datum  can  be 
erroneous  or  completely  absent. 

Phenomenon 

Discretization 

The  discretization,  at  the  point  of  datum  acquisition,  of  a  phenomena  or  its 
attribute  that  is  in  reality  continuous  in  space,  time  or  another  similar 
dimension.  In  other  words,  the  act  of  acquiring  data  at  intervals  for 
something  that  is  continuous.  (Note:  Discretization  (or  sampling)  of  data 
after  data  acquisition  would  fall  under  Transformation.) 

Noise 

Extraneous  background  activity  that  produces  unwanted  influence  on  the 
acquired  data. 

Ambiguous  Data 

Data  that  cannot  be  interpreted  correctly  without  further  information.  The 
ambiguity  could  be  due  to  a  lack  of  reference  or  simply  confusion  as  to 
how  to  interpret  the  data. 

Subjectivity 

Subjective  influence  in  the  data. 

Data  Age 

The  age  of  the  data  from  the  moment  of  acquisition.  This  is  a  cause  of 
uncertainty  only  in  cases  where  the  data  are  time  dependent. 

Data  Corruption 

Corruption  of  data  at  some  point  during  the  transfer-storage-manipulation- 
usage  process  due  to  human  or  machine  error.  This  can  occur  anytime 
after  the  moment  of  data  acquisition  for  the  entire  lifecycle  of  the  data. 

Data  Extrapolation 
&  Interpolation 

Extrapolating  or  interpolating  data  from  indirect  evidence.  The 
extrapolated  or  interpolated  data  has  an  inherent  uncertainty. 

Transformation 

The  changing  of  data  from  their  original  form  or  format. 

Misinterpretation 

A  mistake  made  in  interpreting  the  data. 
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audience  can  distinguish  between  relative  degrees  of  uncertainty;  the  audience  can  extract  values 
of  uncertainty;  etc. 

To  prepare  the  uncertainty  for  visualization  is  to  do  what  is  necessary  to  have  the  required 
uncertainty  ready  to  be  used  in  the  visualization.  This  could  involve  calculating  uncertainty  from 
collected  data,  transforming  uncertainty  measurements  into  the  proper  units,  assigning 
uncertainty,  extracting  uncertainty  from  collected  data,  etc.  The  past  steps  may  have  also 
illuminated  that  the  uncertainty  data  need  to  be  prepared  in  ways  that  were  not  evident  at  the 
outset.  Including  this  step  at  this  point  in  the  strategy  allows  for  this  new  understanding  to  be 
taken  into  consideration  when  preparing  the  data.  For  example,  using  the  previous  ship  example, 
the  multiple  uncertainties  may  need  to  be  properly  combined. 

To  try  different  uncertainty  visualization  techniques  is  the  creative  part  of  this  strategy.  It  is  the 
step  where  the  designer  takes  what  they  have  learnt  from  the  strategy  and  applies  their  own 
creative  ideas,  visualization  theory,  past  experience,  or  techniques  they  have  seen  elsewhere,  to 
create  ways  of  visualizing  the  uncertainty.  It  may  become  obvious  in  this  step  that  the  uncertainty 
data  have  to  be  manipulated  further,  thus  the  previous  step  may  need  to  be  revisited. 

To  get  audience  opinions  and  criticisms  includes  doing  user  studies,  usability  studies,  etc.  This 
step  is  arguably  the  most  important  step  of  the  strategy  because  it  applies  the  visualization  to  a 
user- focused  application.  In  addition,  the  results  of  structured  user  and  usability  studies  can  guide 
future  work.  This  step  will  likely  require  a  revisit  to  the  previous  step  once  results  are  obtained. 

It  is  important  to  note  that  these  last  two  steps  are  essentially  the  “trial  and  error”  method.  One 
goal  of  this  strategy  is  to  minimize  the  trial  and  error  iterations.  In  turn,  this  will  minimize 
designer  and  user  frustrations  while  optimizing  product  delivery  time. 
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3  UVDS  Illustrated  Through  a  Canadian  Recognized 
Maritime  Picture  Uncertainty  Visualization 
Experiment 


In  the  previous  section,  the  Uncertainty  Visualization  Development  Strategy  was  described.  The 
UVDS  is  now  mapped  to  uncertainty  visualization  work  that  has  been  done  with  regards  to  the 
Recognized  Maritime  Picture  (RMP)  created  and  maintained  by  the  Canadian  military’s  Regional 
Joint  Operations  Centres  (RJOCs).  The  UVDS  was  developed  prior  to  this  RMP  uncertainty 
visualization  research  but  not  actively  applied  during  the  research.  Nonetheless,  the  UVDS  can  be 
easily  mapped  to  the  work  done  in  the  project.  Further  information  on  the  details  and  results  of 
this  visualization  research  can  be  found  in  Matthews  et  al.  [24]  and  Matthews  and  Rehak  [25]. 

The  RMP  is  a  tool  used  to  develop  Maritime  Domain  Awareness  (MDA)  for  Canadian  military 
operation  centres.  In  its  common  form,  it  is  represented  as  a  map  of  contacts  (Figure  2),  where  a 
contact  is  defined  as  any  discrete  airborne,  surface  or  subsurface  object  detected  by  electronic, 
acoustic,  and/or  visual  sensors  [26].  The  term  contact  report  is  often  used  when  discussing  the 
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Figure  2:  Unclassified  example  of  the  Recognized  Maritime  Picture  (RMP)  off  the  coast  of  Nova 
Scotia,  Canada.  Green  indicates  land  while  blue  indicates  ocean.  Each  contact  is  indicated 

using  a  symbol  and  descriptive  text. 
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RMP.  A  contact  report  is  information  about  a  contact  at  a  time  of  observation,  usually  containing 
identity  attributes  and  position.  Identify  attributes  for  a  ship  would  include  name,  flag,  the 
Maritime  Mobile  Service  Identity  (MMSI)  number,  etc.  The  RMP  presents  some  of  this 
information  using  visual  cues  while  other  information  requires  “drilling  down”  into  its  metadata. 
This  map  representation  of  the  maritime  situation  allows  various  maritime  organizations  (e.g., 
RJOCs,  Marine  Security  Operation  Centres  (MSOCs))  to  develop  an  understanding  of  current 
maritime  activities  in  their  Area  of  Interest  (AOI)  or  Area  of  Responsibility  (  AOR). 

An  identified  need  for  the  RMP  to  convey  uncertainty  has  been  recognized  through  discussions 
with  those  who  regularly  interact  with  the  RMP.  The  research  is  being  done  in  multiple  phases. 
Phase  one  [24]  included  doing  basic  research  on  visualizing  uncertainty  in  an  RMP-like 
environment  using  human-in-the-loop  experimentation  to  test  designs.  Non-experts  were  used  in 
the  experimentation.  Phase  two  [25]  included  an  investigation  into  the  usefulness  of  the  designs 
in  an  RMP  environment  using  subject  matter  experts.  Subsequent  phases  will  investigate  how  to 
rigorously  calculate  and  assign  uncertainties  to  the  symbology  developed.  In  the  following,  the 
UVDS  will  be  followed  to  illustrate  the  research  and  the  usefulness  of  the  UVDS. 


3.1  Identify  the  uncertainty  visualization  task 

Uncertainty  present  in  the  RMP  data  needs  to  be  visually  represented  for  those  who  use  and  build 
the  RMP. 

Background:  In  its  present  form,  the  RMP  does  not  offer  cues  as  to  the  uncertainty  associated 
with  the  data  being  presented  in  the  RMP.  Uncertainty  in  the  RMP  can  arise  from,  for  example, 
contact  metadata  being  out  of  date,  wrong  or  missing;  false  contact  reports;  the  time-lag  for  each 
contact  being  different;  lack  of  distinction  between  areas  with  no  contacts  as  compared  to  areas 
with  no  sensor/information  coverage;  and  variations  in  the  sensor/information  coverage  due  to 
varying  weather,  sensor  altitude  (e.g.,  when  mounted  on  a  plane),  ships  not  reporting  as  they 
should,  time  of  day,  etc.  This  underlying  uncertainty  can  mean,  for  example,  that  two  contacts 
side  by  side  in  the  RMP  may  in  reality  not  be  anywhere  near  each  other.  This  situation  is 
common  when  the  contact  positional  information  have  different  associated  time-lags.  It  is  also 
easy  to  wrongly  think  that  the  contact  report  information  for  each  contact  is  equally  believable, 
and  that  where  there  is  an  absence  of  a  contact,  no  object  exists. 


3.2  Understand  the  data  that  need  to  have  their  uncertainty 
visualized 

Each  contact  has  an  associated  set  of  metadata  which  can  include  (but  is  not  exclusive  to)  speed, 
heading,  name,  MMSI  number,  threat,  flag,  destination,  origin,  type,  cargo  and  a  digital  image. 
Each  contact  has  various  degrees  of  metadata  included  in  its  contact  report(s).  At  worst,  there  is 
no  metadata  and  all  that  exists  is  a  position  (i.e.,  there  is  something  at  this  point  in  space).  At  best 
the  metadata  consists  of  all  of  the  above,  and  perhaps  more.  The  different  degrees  of  metadata 
are  due  to  the  multiple  sources  of  data  and  information  that  feed  the  RMP.  These  sources  include 
radar,  surveillance  flights,  self  reporting  systems,  voluntary  reports,  etc.,  each  providing  its  own 
set  of  data  and/or  information.  The  data  and  information  being  received  can  be  in  various 
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formats;  e.g.,  text,  numerical,  and  pixels.  An  in-depth  description  of  all  the  data  and  information 
feeding  the  RMP,  including  the  precision  to  which  the  data  is  received  and  which  data  are  prone 
to  mistakes,  is  not  relevant  to  this  paper.  However,  we  can  suggest  a  fictitious  example  to 
illustrate  the  complexity  of  the  situation  and  what  needs  to  be  accomplished  in  this  step. 

Suppose  we  have  a  source  that  provides  data  to  the  RMP.  This  source,  known  as  Source  X, 
automatically  reports  ship  positions  and  ship  names  every  30  minutes.  Non-commercial  fishing 
vessels  and  pleasure  crafts  in  Canadian  waters  are  mandated  to  carry  a  Source  X  transmitter.  On 
a  clear  sunny  day,  the  transmitter  needs  to  be  within  1 00  nautical  miles  (plus  or  minus  a  nautical 
mile)  of  a  land  based  receiver  for  the  report  to  be  received.  There  are  10  such  receivers  along  the 
east  coast  of  Canada.  The  range  can  decrease  to  50  nautical  miles  during  rain  and  snow  events. 
The  range  also  decreases  during  other  inclement  weather  but  no  reportable  pattern  has  been 
observed.  The  ship  position  is  determined  through  the  global  positioning  system  (GPS)  and  is 
exact  to  within  15  m.  The  position  is  reported  in  latitude  and  longitude  to  1/10000  minute.  The 
name  of  the  ship  is  typed  into  the  Source  X  transmitter  by  personnel  on  the  ship  and  therefore  can 
be  prone  to  misspelling,  shortened  versions  of  the  actual  name  or  the  name  not  being  changed 
when  the  Source  X  transmitter  moves  to  another  ship.  Source  X  reports  nothing  else  about 
the  ship,  meaning  information  such  as  its  destination,  vessel  type  and  flag  are  unknown.  The 
dynamic  information,  such  as  average  speed  and  heading,  can  only  be  approximated  by  receiving 
multiple  reports.  The  Source  X  receivers  send  the  data  to  the  centres  responsible  for  creating  the 
RMP  (e.g.,  the  RJOC)  via  Ethernet,  as  soon  as  the  receiver  receives  a  report.  The  reports  are  not 
in  a  format  that  can  be  automatically  inputted  into  the  RMP  database;  therefore,  the  data  need  to 
be  transformed  into  the  proper  format  when  they  arrive  at  the  RJOC.  The  time-delay  between  a 
Source  X  report  being  sent  by  the  transmitter  and  then  finally  inputted  into  the  RMP  database  is 
approximately  10  minutes. 

The  RMP  has  a  vast  amount  of  data  and  information  of  various  types  from  various  sources.  The 
above  only  describes  one  fictitious  data  source,  but  illustrates  the  potential  complexity  of  the 
RMP  data.  The  UVDS  step  of  understanding  the  data  is  useful  because  it  reminds  (and  requires) 
the  designer  to  become  familiar  with  the  collection  methods  and  the  types  of  data.  The  designer 
must  have  this  knowledge  in  order  to  appropriately  identify,  calculate  and  visualize  the 
uncertainty  as  well  as  explain  the  primary  data  to  which  the  uncertainty  is  related. 

3.3  Understand  why  uncertainty  needs  to  be  visualized  and 
how  the  uncertainty  visualization  needs  to  help  the  user 

In  the  case  of  the  current  RMP,  there  is  nearly  nothing  to  signal  to  the  viewer  that  there  is 
uncertainty  in  what  they  are  viewing  (e.g.,  see  Figure  2).  An  RMP  with  none  of  the  uncertainty  in 
its  contacts  displayed  looks  like  all  the  information  is  current,  all  the  ships  are  currently  in  the 
positions  displayed  on  the  map,  all  the  information  comes  from  sources  that  never  have  mistakes, 
and  the  entire  area  is  monitored  by  sensors  feeding  the  RMP.  However,  understanding  the 
uncertainty  in  the  picture  would  be  useful  for  those  maintaining  the  RMP  (i.e.,  the  operators),  for 
those  trying  to  understand  the  picture,  and  for  those  doing  analysis  tasks  based  on  its  content.  For 
example,  for  an  informed  decision  to  be  made  regarding  the  RMP,  whether  it  is  regarding  where 
to  send  a  surveillance  asset,  how  to  prioritize  the  order  a  group  of  ships  should  be  investigated, 
etc.,  more  information  on  the  uncertainty  surrounding  the  contacts  in  the  RMP  needs  to  be 
known.  While  the  uncertainty  levels  can  be  determined  by  investigating  each  contact  one  by  one, 
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that  is  a  time  consuming  process  if  there  are  more  than  a  few  ships  to  be  investigated.  Having  a 
visual  representation  of  the  uncertainty  overlaid  onto  the  RMP  can  help  a  decision  maker  quickly 
assess  a  situation  and,  if  necessary,  decide  on  which  contacts  or  areas  are  worth  a  closer  look 
before  a  decision  is  made.  The  uncertainty  visualization  needs  to  give  those  who  use  and  build  the 
RMP  the  ability  to  quickly  understand  the  uncertainty  in  the  RMP  with  minimal  cognitive  effort. 

3.4  Decide  on  the  uncertainty  to  be  visualized 

Discussions  with  those  who  work  with  the  RMP  identified  three  primary  uncertainties  which 
require  visualization:  uncertainty  due  to  the  passage  of  time;  uncertainty  in  position;  and 
uncertainty  in  identity  (i.e.,  who  the  contact  is). 

Every  piece  of  RMP  metadata  has  uncertainty.  Without  this  step  to  narrow  down  the  potentially 
useful  uncertainty  to  be  visualized,  effort  could  be  wasted  trying  to  visualize  unimportant 
uncertainty. 

3.5  Decide  on  a  definition  of  uncertainty 

Based  on  the  definition  given  in  Plewe  [23],  uncertainty  was  defined  as  follows:  uncertainty  is 
the  dissimilarity  between  a  given  representation  of  reality  and  the  known  or  unknown 
reality,  where  the  unknown  reality  simply  means  you  do  not  know  what  the  reality  actually  is  that 
you  are  representing. 

This  is  a  broad  definition  that  incorporates  many  definitions  of  uncertainty.  It  was  an  appropriate 
definition  for  the  task  at  hand.  Without  this  step,  someone  may  assume  that  the  uncertainty  being 
represented  includes,  for  example,  “feelings”  of  uncertainty.  It  does  not,  unless  those  feelings  are 
based  on  provable  observations  of  dissimilarity  between  reality  and  the  representation  of  reality. 

3.6  Determine  the  specific  causes  of  the  uncertainty 

Uncertainty  in  the  RMP  may  be  specifically  caused  by  the  time  between  source  reporting; 
precision  and  accuracy  of  the  data;  information,  data  and/or  metadata  being  wrong  or  missing, 
human  or  machine  error;  false  contact  reports;  malfunctioning  information  or  data  source;  the 
geographical  area  not  being  surveyed  either  recently  or  at  all;  the  weather;  ships  not  reporting  as 
they  should;  time  of  day;  changes  in  attributes  of  the  contact  between  reports;  etc.  The  causes  of 
uncertainty  in  the  RMP  have  the  potential  of  being  the  cause  of  uncertainty  due  to  the  passage  of 
time,  uncertainty  in  position  and  uncertainty  in  identity,  identified  in  Section  3.4  as  the 
uncertainty  to  be  visualized. 

Note  that  uncertainty  due  to  the  passage  of  time  and  uncertainty  in  position  are  partially  linked. 
Part  of  the  uncertainty  in  position  is  due  to  the  passage  of  time,  combined  with  the  uncertainty  in 
the  speed  and  in  the  direction  of  the  ship.  Uncertainty  in  position  can  also  be  caused  by  the 
reliability,  the  precision  and  the  accuracy  of  the  source  of  the  information. 

This  step  illuminates  the  many  causes  of  uncertainty  present  in  the  RMP  data  and  therefore  the 
potential  causes  of  the  uncertainties  the  designers  wish  to  focus  on,  which  is  important  for  the 
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designer  to  understand.  It  also  provides  the  designer  needed  information  for  later  calculating  the 
uncertainty  in  preparation  for  visualization. 

3.7  Determine  the  causal  categories  of  the  uncertainty 

Based  on  Table  1,  the  (collective)  causal  categories  of  the  three  uncertainties  to  be  visualized  are 
Data  Acquisition  Limitation  (e.g.,  GPS  measurement  accuracy);  Data  Acquisition  Error  (e.g., 
mistyping  information);  Phenomenon  Discretization  (e.g.,  position  of  a  moving  ship);  Noise; 
Ambiguous  Data  (e.g.,  using  the  short  form  for  a  ship  name);  Data  Age;  and  potentially  Data 
Corruption  and  Misinterpretation. 

This  step  helps  categorize  the  specific  causes  of  uncertainty  identified  in  the  previous  step  into 
much  more  manageable  groupings.  These  groupings  also  help  the  designer  understand  the 
generalized  problems  associated  with  the  data,  and  potentially  aid  in  the  design  of  more  general 
solutions. 


3.8  Determine  the  visualization  requirements 

For  this  application,  the  visualization  should  give  the  audience  enough  information  to  make  an 
educated  decision  on  whether  to  obtain  more  information  about  a  contact,  group  of  contacts  or 
area.  The  audience  must  be  able  to  distinguish  between  degrees  of  uncertainty  but  there  is  no 
need  for  the  audience  to  be  able  to  extract  the  values  of  the  uncertainty  from  the  visualization. 
The  uncertainty  representation  is  to  be  presented  in  conjunction  with  the  contact  symbols  and 
must  not  overwhelm  the  contact  symbol  or  distract  from  it.  The  uncertainty  representation  must 
also  avoid  using  characteristics,  such  as  the  colour  red,  that  might  have  unintended  meanings 
associated  with  them.  Clutter  should  also  be  minimized.  The  following  is  a  modified  list  of  the 
visualization  requirements  stated  in  [24]: 

•  The  uncertainty  representations  must  not  (unintentionally)  interact  with  each  other. 

•  Each  uncertainty  level  must  be  uniquely  understood  and  not  rely  on  the  presence  of 
other  uncertainty  levels. 

•  The  visual  characteristics  must  maintain  their  ability  to  be  discriminable  under  a  range 
of  ambient  lighting  conditions. 

•  The  use  of  colours  and  symbols  must  not  conflict  with  RMP  symbology. 

•  The  representation  must  be  understood  immediately  without  requiring  additional 
actions. 

•  The  size  of  the  symbology  must  be  appropriate  for  the  zoom  level. 

•  The  meaning  of  the  coding  must  not  require  the  operator  to  rely  on  long  term  memory  or 
consult  supplementary  information  sources. 

•  Wherever  possible,  the  coding  should  conform  to  existing  psychological  constructs  and 
stereotypes. 

•  Simpler  symbol  designs  are  preferred  over  more  complex  designs. 
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•  Use  of  alphanumeric  characters  is  not  preferred. 

•  Blinking  or  flashing  symbology  encoding  must  not  be  used. 

•  Use  of  blurring  of  symbols  (e.g.,  [6])  must  be  avoided. 

This  step  is  indispensable  to  ensure  that  the  visualization  is  useful  and  practical.  The  physical 
environment  in  which  the  RMP  is  used  and  maintained,  as  well  as  in  what  way  the  RMP  is  used, 
maintained,  and  represented,  must  be  taken  into  consideration. 

3.9  Prepare  the  uncertainty  for  visualization 

For  the  initial  phase’s  experimentation,  the  uncertainty  levels  were  assigned  randomly  because 
the  objective  of  the  experiment  was  to  test  the  initial  symbology  (1.  &  2.  below)  and  thus  did  not 
require  a  more  comprehensive  assignment  of  uncertainty  values.  Then,  in  phase  two  of  the 
investigation,  the  uncertainty  levels  were  assigned  using  observed  source  reliability  and  how 
operators  treat  data  based  on  age  (3.  &  4.  below).  A  complete  calculation  of  the  uncertainties  of 
the  data  being  looked  at  is  a  non-trivial  exercise.  Finding  rigorous  ways  to  calculate  the 
uncertainties  of  interest  is  a  future  phase  of  this  work. 

This  step  is  essential  for  creating  an  uncertainty  visualization.  The  choice  to  avoid  explicitly 
calculating  uncertainty  at  this  moment  in  the  research  allowed  the  research  to  progress  without 
being  obstructed  by  the  complexity  of  the  uncertainty  calculations. 

3.10  Try  different  uncertainty  visualization  techniques  and  get 
audience  opinions  and  criticisms 

(Due  to  the  loop  between  the  final  two  tasks,  they  will  be  treated  in  one  subsection.) 

1.  At  first,  in  phase  one,  the  “audience”  were  those  working  on  this  project:  two  defence 
scientists  from  Defence  Research  &  Development  Canada  (DRDC)  and  the  team  of 
contractors  from  Yhxmansystems  Inc.  (HSI)  [24].  Different  visualization  techniques  were 
designed  and  then  the  audience  would  give  their  opinions  and  criticisms.  There  were  many 
iterations  of  this  cycle  before  a  final  set  of  visualizations  were  developed  for  formal  testing. 

The  final  set  is  shown  in  Figure  3  and  are  labelled  the  Lego1  Design  and  the  Rectangle 
Design  for  convenience.  The  visualization  requirements  of  Section  3.8  greatly  limited  the 
design  options  of  the  uncertainty  representation.  The  size  of  the  RMP  contact  symbols,  which 
are  of  the  order  of  5  mm  on  the  RMP  operator’s  computer  monitor  (much  smaller  than  shown 
in  Figure  3),  was  arguably  the  biggest  limiter  of  potential  options.  Uncertainty  in  identity  is 
represented  by  the  colour  of  the  blocks  (Lego  Design)  and  the  rectangle  (Rectangle  Design). 
Uncertainty  due  to  the  passage  of  time  is  represented  by  the  number  of  vertical  blocks  (Lego 
Design)  or  the  position  of  the  black  circle  (Rectangle  Design).  Uncertainty  in  position  is 
represented  by  the  number  of  horizontal  blocks  (Lego  Design)  or  the  radius  of  the  black  circle 
(Rectangle  Design). 


1  This  was  a  label  of  convenience  based  upon  the  similarity  of  the  design  to  LEGO  building  blocks  and  is 
not  associated  with  the  registered  trademark  of  The  LEGO  Group. 


DRDC  Atlantic  TM  2009-151 


13 


As  the  uncertainty  representations  merely  needed  to  cue  the  viewer  to  the  presence  and  extent 
of  the  uncertainty,  the  team  chose  to  use  colour,  spatial  position  and  size  to  represent  levels  of 
uncertainty.  For  each  aspect,  (i.e.,  identity,  time  and  position)  there  were  three  levels  of 
uncertainty  that  could  be  represented.  The  colour  coding  used  to  represent  uncertainty  in 
identity  uses  a  modified  traffic  signal  convention;  however,  the  colour  red  could  not  be  used 
due  to  conflicts  with  other  colour  coding  in  the  RMP.  In  the  Rectangle  Design,  the  team  also 
chose  to  represent  temporal  uncertainty  increasing  from  left  to  right  because  there  was  no 
clear  convention  as  to  whether  increasing  left  to  right  or  right  to  left  would  be  more 
intuitively  understood.  The  team  also  chose  to  represent  increasing  positional  uncertainty 
with  increasing  horizontal  blocks  (Lego)  and  dot  diameter  (Rectangle)  to  emulate  the  already 
established  convention  that  big  error  ellipses  around  a  position  means  large  uncertainty. 


Lego  Design 


Good  Identity 
Current  Time 
Precise  Position 


Poor  Identity  No  Identity 

Old  Time  Out  of  Date  Time 

Approximate  Position  Unknown  Position 


Rectangle  Design 

Good  Identity 
Current  Time 
Precise  Position 


Poor  Identity  No  Identity 

Old  Time  Out  of  Date  Time 

Approximate  Position  Unknown  Position 


Figure  3:  An  enlarged  example  of  the  two  designs  of  uncertainty  symbology  tested  in 
phase  one  of  the  RMP  research.  The  blue  octagon  represents  the  contact.  Uncertainty  in 
identity  is  represented  by  the  colour  of  the  blocks  (Lego  Design)  and  the  rectangle 
(Rectangle  Design).  Uncertainty  due  to  the  passage  of  time  is  represented  by  the  number 
of  vertical  blocks  (Lego  Design)  or  the  position  of  the  black  circle  (Rectangle  Design). 
Uncertainty  in  position  is  represented  by  the  number  of  horizontal  blocks  (Lego  Design) 
or  the  radius  of  the  black  circle  (Rectangle  Design).  Figure  taken  from  [24] . 


2.  The  next  set  of  audience  members  were  formal  experiment  subjects,  as  described  in  [24], 
The  experiment  tested  the  uncertainty  visualization  symbology  using  a  combination  of 
training,  timed  taskings,  and  surveys.  This  concluded  phase  one  of  the  work.  Accuracy  was 
high  for  both  designs,  with  no  significant  differences  between  them. 

3.  In  phase  two  [25],  the  audience  members  were  initially  made  up  of  members  of  DRDC  and 
HSI  who  were  working  on  the  project.  The  symbology  used  in  the  previous  experiment  was 
imported  into  another  environment,  Google™  Earth  (GE),  since  many  other  operator  tools  are 
now  being  released  in  GE.  There  were  certain  aspects  of  the  symbology  that  needed  to  be 
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altered,  such  as  colour,  weight  of  lines,  size,  etc.  Several  iterations  of  opinions  and  criticisms 
were  required  to  come  up  with  a  satisfactory  symbol  set  in  this  new  environment.  A  partial 
example  is  shown  in  Figure  4. 

4.  The  next  set  of  audience  members  were  subject  matter  experts  (SMEs);  i.e.,  those  who  use 
and  maintain  the  RMP.  These  were  again  formal  experimental  subjects.  They  were  given 
training  and  then  tasks  to  accomplish  using  the  visualizations  decided  upon  in  the  previous 
step.  A  questionnaire  was  also  administered.  The  results  helped  gauge  the  usefulness  of  the 
symbology.  The  pattern  of  the  SME  performance  data  was  very  similar  to  the  pattern  from 
the  previous  experiment  with  the  non-experts.  The  subjective  questionnaire  evaluations 
showed  strong  support  for  the  usability  of  the  aids  and  there  was  general  belief  that  they 
would  add  to  operational  capability. 


Figure  4:  Examples  of  the  symbology >  imported  into  Google™  Earth,  appearing  with 

contact  symbology. 


The  final  two  steps  in  the  strategy  built  upon  all  the  previous  steps.  The  steps  were  essential  in 
narrowing  down  potential  visualizations  and  judging  them  with  respect  to  the  RMP.  The  next 
step  in  this  research  will  be  to  revisit  “prepare  the  uncertainty  for  visualization”  and  determine 
some  rigorous  methods  to  dynamically  calculate  the  uncertainty  that  feeds  the  symbols. 

3.11  RMP  application  remarks 

Section  3  has  shown  that  the  UVDS  provides  a  systematic  method  to  design  uncertainty 
visualizations.  The  large  number  of  data  and  information  sources  continuously  feeding  the  RMP 
result  in  an  abundance  of  uncertainty  being  imbedded  in  the  RMP.  The  RMP  example  shows 
how  each  step  can  help  navigate  the  designer  towards  producing  a  useful  uncertainty 
representation  by  requiring  them  to  complete  the  eleven  steps.  It  has  also  been  shown  here  that 
following  the  steps  to  document  the  work  can  help  explain  the  research. 
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4  Concluding  Remarks 


The  strategy  discussed  here  has  been  created  specifically  to  help  the  designer  better  understand 
what  steps  need  to  be  accomplished  to  produce  uncertainty  visualizations.  The  eleven  steps  are 
all  logical  and  practical  steps  that  may  have  been  addressed  naturally  by  a  designer  who  is 
unaware  of  the  UVDS.  There  is  benefit  to  formalizing  the  steps  both  for  the  novice  and 
experienced  designer.  For  the  novice,  the  steps  provide  documented  guidance  for  understanding 
the  process  to  be  followed.  For  the  experienced  designer,  it  provides  reminders  for  steps  which 
should  be  explicitly  thought  about.  In  addition,  each  finished  step  could  serve  to  help  document  a 
finished  product,  which  can  be  used  when  discussing  or  justifying  a  design  or  giving  in-depth 
background  on  the  visualization,  as  shown  in  Section  3. 

Further  work  on  the  UVDS  would  be  to  incoiporate  changes  suggested  by  the  users  of  the 
strategy.  UVDS  could  also  be  augmented  as  formal  uncertainty  visualization  theory  is  developed, 
thus  providing  guidance  on  what  uncertainty  visualization  techniques  to  implement  given  the  type 
of  uncertainty  to  be  visualized  (e.g.,  [8]).  Meanwhile,  the  UVDS  could  work  in  conjunction  with 
existing  theories,  postulates  and  frameworks. 
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List  of  symbols/abbreviations/acronyms/initialisms 


AOI 

Area  of  Interest 

AOR 

Area  of  Responsibility 

DND 

Department  of  National  Defence 

DRDC 

Defence  Research  &  Development  Canada 

DRDKIM 

Director  Research  and  Development  Knowledge  and  Information 
Management 

GE 

Google™  Earth 

GPS 

Global  Positioning  System 

HS1 

1  Iuman.vvvte/7?.v  Inc. 

MDA 

Maritime  Domain  Awareness 

MMSI 

Maritime  Mobile  Service  Identity 

MSOC 

Marine  Security  Operation  Centre 

R&D 

Research  &  Development 

RJOC 

Regional  Joint  Operations  Centre 

RMP 

Recognized  Maritime  Picture 

SME 

Subject  Matter  Expert 

UVDS 

Uncertainty  Visualization  Development  Strategy 
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